Xiden/Guix design comparison

[Sage Gerard <sage@sagegerard.com>]

[-- Attachment #1: Type: text/plain, Size: 14454 bytes --]

This is an offshoot Q&A based off Devos' questions in the "How did you handle making a GNU/Linux distribution?" thread. It pertains to how Guix compares to Xiden on some points.

Given the length of both of our emails, I don't expect everyone to read this. Guix is mentioned for discussion purposes, but I use the -devel list only because the questions did. Apologies to the mods if that's not an excuse, since I'm not trying to distract from the purpose of the list.

Read on only if you care.

--

> Packages in Guix can specify which version of a package they use. [...] Is something missing from Guix here?

In Xiden, package versions and the notations used to express them are user-defined. If you use semver and I use dates, we can still map our dependency lists to the same package definitions. Xiden leaves it to us to agree on a scheme for some set of packages.

> SHA-384 and Keccak are separate things? [...] By definition, implementations of SHA-384 have no reason to use Keccak.

Correct. I'm just saying that Alice's decision to prefer a SHA-384 implementation for using Keccak is not our business. Ditto for Bob and SHA-1. Alice can force use of Keccak for software arriving on her system with Xiden, without needing to discuss that with anyone.

She's in charge of how a package manager works on her system, and we aren't. Xiden approaches CHFs similarly to OpenSSL in that implementations are selected by providers, except in Xiden a knowledgeable user may assume the role of a CHF implementation provider for their own system. I suspect we'd agree that is a terrible idea, which is why I use zero-trust defaults for everything to mitigate risks.

I allow for danger because most software distribution problems I've seen in the field boil down to smart people dealing with the lasting consequences of outdated or rushed decisions. I'm not assuming competency, but I am assuming that even those with bad practices still need to solve their distribution problems.

>> unconditionally prefers cached installations, such that every defined artifact
>> has exactly one stored copy on disk.
>
> Coming from Guix, I don't know what this means. Does this mean only one version
> of a package can be in the store at the same time, and building a newer version
> deletes the older version? How does this compare with ‘content deduplication’?

Xiden has a cache for package inputs, and a cache for installed outputs. The user has more control over the latter. Depending on the configuration you use in a launcher, you can map all inputs to the same content-addressable filesystem, implying deduplication. However, you can also decide to defeat the installed output cache for the sake of how SxS installations in Xiden. That way you can store any number of any revision of any package in one store (Although I use the word "state" in the docs).

>> but only for artifacts from services authenticated by his operating system's certificates
>
> This is rather vague to me, unless you're referring to substitutes.
> What does this mean for origin specifications?
>
> (source (origin
> (method url-fetch)
> (uri (string-append ["mirror://gnu/hello/hello-"](mirror://gnu/hello/hello-) version
> ".tar.gz"))
> (sha256 ; imagine this was SHA-1 instead.
> (base32
> "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))
>
> Bob got the origin specification from Xiden (or something like a Guix channel
> but for Xiden). How would this ‘artifact’ authenticated by ‘operating system
> certificates’? I mean, when you let your operating system build that origin
> (via guix (or Xiden?), which you could consider part of the OS), it will check
> that hash, and it will be considered valid (‘authenticated?’) if the hash matches.
> There are no certificates at play here.
>
> I suppose the OS (guix or Xiden?) could then ‘sign’ it, but what value does that
> provide to Bob? (Unless Bob runs a substitute server on their computer.)

Xiden distinguishes authenticating an artifact and authenticating a service from which an artifact is specified. I mentioned the latter, so I was talking about SSL or TLS authentication.

The word "source" means something different in Xiden. It refers to a structure instance that implements a titular Racket generic. When "tapped", a Xiden source yields an input port and an estimated number of bytes available for the port, or +inf.0 if no estimate is available. The user would tell a launcher what budgets it would allow, if any, and Xiden will only read bytes when the budget is in agreement with the estimate.

When I say "artifact", I mean one of two things.

- An (artifact) expression in a Xiden package input, which consists of a Xiden source, integrity information, and signature information (where available).
-  Some blob of data that I can verify after a download. Definition #1 is just a more restricted form of the same idea.

So to help us discuss this point I'd have translate your expression to a package input in Xiden as follows. I swapped the server and added a signature expression to clarify something.

(input "hello"
(artifact (http-source (string-append ["https://packages.example.com/hello/"](https://packages.example.com/hello/) version ".tar.gz"))
(integrity 'sha1 (base32 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))
(signature (http-source ["https://packages.example.com/public-key.pem"](https://packages.example.com/public-key.pem))
(http-source (string-append ["https://packages.example.com/hello/"](https://packages.example.com/hello/) version ".tar.gz.sha1.sig")))))

I was saying that Bob can trust the CA that signed the certificates for packages.example.com, but handwaved over that because I was already writing such a long email.

The content of the .tar.gz file is authenticated separately. You can see that I happened to use a PEM-encoded public key and a remote signature file. The end-user's launcher would have to be compatible. If a user prefers to use GPG and not allow for signatures that sit outside of a package definition, their launcher would say so.

There's an example in Xiden's source that shows how to swap out cryptographic backends, so if you don't want to use OpenSSL or GPG, or if you want to automate conversions of formats between the two, the option is available.

And if some of this sounds vague, that's because it is. Semantics of some forms are not fully defined without a launcher. e.g. Xiden only understands the (signature) form to authenticate data using assymetric cryprography, without caring about how exactly you'd do it.

>> trusts digest creation and signature verification as implemented by his host's dynamically--linked `libcrypto` library for use in the same process.
>> downloads content from GitHub packages
>
> AFAIK there is no such thing a ‘GitHub packages’. What is a ‘GitHub package’,
> precisely?

I was referring to the output of the packages feature Microsoft rolled out for GitHub. It integrates with the GitHub Actions feature for CI/CD.

I am actually going to prepare an example where I use a GitHub repository as a catalog of sorts, such that pushing to the repository adds a valid target for a launcher.

>> such that packages cannot conflict in a shared namespace
>
> ‘packages cannot conflict in a shared namespace’ is a bit vague.
> Are you referring to ‘profile collisions’? About having multiple versions
> of the same package on the same system? Guix (and nix) can handle the latter.
> When the ‘propagated-inputs’ are limited, installing multiple packages each using
> a different version of a package is possible.
>
> Is something missing from Guix here (please be very concrete)? [...] I'm not sure what you mean here, can you illustrate this very concretely? And is something missing in Guix here?

I am not in a position to speak for others on what technical aspects are missing or inferior in Guix. That's not what I'm trying to do here.

I only care about how easily a human being can switch between concrete details in this problem domain, which means Xiden leaves some details for the user to define. I'll give you concrete details on how I try to enable this, and I'm not trying to say Guix is missing anything by doing so.

One detail to note is that Xiden does not have profiles. With an unprivileged user, Xiden launchers all act like `ln -s <totally-made-up-notation> link-name`. I didn't think it needed to be more complicated than that.

>> prefers Semantic Versions
>
> If upstream doesn't do ‘semver’, and instead uses a versioning system like
> used by TeX (3.14, 3.141, 3.1415, 3.14159, ...), then Xiden cannot somehow
> let the package use ‘semver’ instead. Unless you want to teach Xiden to
> convince upstream to change their versioning system?

A Xiden launcher may translate your preferred version notation to an upstream scheme.

>> These preferences are valid in Xiden's DSL for writing launchers.
>
> What is a launcher (compared to packages in Guix), and why would I want one?

The docs say what a launcher is.

You want one if you want to adapt to social and technical differences between package management systems. Xiden does not know what you want without clarification from you, for similar reasons to why Vulkan ended up more explicit than OpenGL.

> Why shouldn't I just install packages and run them ("guix package -i texmacs",
> start texmacs from some application chooser menu of the desktop environment).

Your question assumes `guix` is the command you want to run, which is fine, but not everyone is you. I suspect that's why you are struggling to understand how Xiden works, because Xiden does not prescribe itself as the sole option for how you put things on your system. It can also serve as a bridge between your existing options.

I think the pattern of everybody and their cat making a GNU/Linux distribution with a new package manager raises questions about interoperability. I'm trying to address that by saying that Xiden is to package managers what Racket is to programming languages. The white paper covers this.

>> The invariants for each launcher can differ as much as Cargo and Vortex differ.
>> What I wanted to do was allow users to declare their own invariants explicitly,
>> but only when those invariants are wanted for subjective reasons.
>
> What are these invariants you're speaking of? I don't think you're referring
> to, say, translation invariance (‘the laws of physics remain the same under a
> translation of the coordinate system’).
>
> Or do you mean something like ‘Invariant (computer science), an expression whose
> value doesn't change during program execution’ ([<https://en.wikipedia.org/wiki/Invariant>](https://en.wikipedia.org/wiki/Invariant))?

https://stackoverflow.com/a/112088

> But what are the ‘expressions’ in this case?

If I don't mean S-expressions, I am probably referring to a string of arbitrary content with a meaning made up by the user.

>> Xiden launchers have advantages over shell scripts in that we can both still download
>> and install dependencies from the same servers, and create reproducible builds in terms
>> of the same (bit-identical) state on disk. We just differ on details that shouldn't impact
>> how we work. It also helps us patch holes faster, since if (say) a catalog maintainer tells
>
> What is a ‘catalog maintainer’ (maybe it's like someone with commit access to a ‘guix channel’?)

A catch-all term I use for a person with a server that responds with artifacts, by definition #2 of "artifact."

> Also, what ‘shell scripts’ are you referring to.

Doesn't matter. That bit was me anticipating questions like "Why go this abstract when I could just write a shell script?"

>> us that a CHF was compromised, we can immediately revoke trust in the CHF independently
>
> Maybe guix could gain a command "guix style transition-hash OLD NEW" to automagically
> rewrite package definitions to use the new hash algorithm instead of the old. It will
> need to download plenty of source code however, and it will entail a world-rebuild.

Xiden doesn't have that problem. Or, more accurately, it doesn't take responsibility for that problem. I think the real world is too messy to try that.

>> in our own clients.
>
> What are these ‘clients’ you're speaking of? I haven't been needing any ‘clients’ lately.

In the context of Xiden, "client" may be a synonym for "launcher" if the launcher uses the network.

>> The package definitions are all expressed in terms of what the launchers allow.
>
> If 'package definition' = (package definition as in Guix), then this is false,
> Guix doesn't have a notion of ‘launchers’ and doesn't depend on ‘Xiden’.
> What are you meaning, exactly? Also, I haven't found any package definitions in Xiden,
> could you point me to any?

The guide shows a command that prints where you can find examples. I have spent over an hour typing before getting to this question, so I ask that you please read the docs more.

If you are looking for a definition for something like GNU coreutils, then you won't find one without an associated launcher. I'll get to launcher details in your other questions. I am not currently hosting any system-level packages, so you won't get definitions from me.

> Why would I want to ‘escape to a new distribution model’? Writing something like Guix
> from scratch seems a lot of work to me, I would rather fork Guix (with a few
> likewise-minded collaborators) than to start over again. Or move to Debian maybe.
> I don't see what ‘Xiden’ could offer here.

The details that would prompt one to fork a package manager source tree can be trivially changed in a Xiden launcher.

> I would rather not have to ‘adapt to problems’, I would rather that upstream has something
> that works (though I can understand if occasionally mistakes are made).

We differ here. I need the ability to make my own decisions that may contradict upstream decisions at any time. That's a freedom I value and aim to protect.

--

I didn't bother with the rest of the email because I don't have time to repeat answers found elsewhere, or define every other word. There are enough contextual clues to help your own research. I'm happy to take further questions, but please ask with the understanding that I've already been generous with my time here. If there's a problem with Xiden's approach, please open a GitHub issue against zyrolasting/xiden.

Thanks.

[-- Attachment #2: Type: text/html, Size: 23566 bytes --]