From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mp0 ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by ms0.migadu.com with LMTPS id +F3uCw5MJmF4JAAAgWs5BA (envelope-from ) for ; Wed, 25 Aug 2021 15:56:30 +0200 Received: from aspmx1.migadu.com ([2001:41d0:8:6d80::]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) by mp0 with LMTPS id IDiqBw5MJmHyZAAA1q6Kng (envelope-from ) for ; Wed, 25 Aug 2021 13:56:30 +0000 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by aspmx1.migadu.com (Postfix) with ESMTPS id 73A59265BD for ; Wed, 25 Aug 2021 15:56:25 +0200 (CEST) Received: from localhost ([::1]:34374 helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1mItOC-0001Qa-HX for larch@yhetil.org; Wed, 25 Aug 2021 09:56:24 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:41184) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1mInV2-0001O6-Ms for guix-devel@gnu.org; Wed, 25 Aug 2021 03:39:04 -0400 Received: from mail.arctype.co ([138.68.9.245]:39805) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1mInUx-0007Xw-P0 for guix-devel@gnu.org; Wed, 25 Aug 2021 03:39:04 -0400 Received: from authenticated-user (mail.arctype.co [138.68.9.245]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)) (No client certificate requested) by mail.arctype.co (Postfix) with ESMTPSA id 0A24B13B238; Wed, 25 Aug 2021 07:38:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=arctype.co; s=mail; t=1629877104; bh=SWXADP+QE7g8NZLAcUVjLxkeKpq96kTKooehMTjlBic=; h=Date:In-Reply-To:References:Subject:To:From:From; b=a4jjPriZONDU2xn/Og8MfT8vAc/H6xqOoRnlZu0/vgZsDRWaOR3j650OxyqR+mzkC Lt5C3fRiQeNDePcXE5ucd7AN0c6OwCtQLNoJHDkRxW9EF0p2PwOjL2czQbv4bud60P 5U3923p9V6ltKf1h6iwOGaVWqJVMxR1WgYJaWLmFX+tErYRDXTmONmHXn/5gTJ3NGO n/4Jbi2963z3PppvXD7ItTIV0gBrXF0/tbLhgJ9tHfB+gK0v0nltpXk+Ap1heWXvOC sIjAMToK9J43MU08IuSgFNyZvK55D5chOJrFd6JxXzgxG6YmgKa7uuFNe/13vftibd r298bmJFJ73TA== Date: Wed, 25 Aug 2021 00:38:22 -0700 In-Reply-To: <51708424-e8c8-80aa-32bd-a5e71a891a71@sagegerard.com> References: <51708424-e8c8-80aa-32bd-a5e71a891a71@sagegerard.com> MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----IFJP6XIDEHQZPO8J3Q6QTYMCO2DSAD" Content-Transfer-Encoding: 7bit Subject: Re: Xiden/Guix design comparison To: Sage Gerard ,guix-devel@gnu.org From: Ryan Sundberg Message-ID: Received-SPF: pass client-ip=138.68.9.245; envelope-from=ryan@arctype.co; helo=mail.arctype.co X-Spam_score_int: -20 X-Spam_score: -2.1 X-Spam_bar: -- X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, HTML_MESSAGE=0.001, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Mailman-Approved-At: Wed, 25 Aug 2021 09:56:04 -0400 X-BeenThere: guix-devel@gnu.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+larch=yhetil.org@gnu.org Sender: "Guix-devel" X-Migadu-Flow: FLOW_IN ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=yhetil.org; s=key1; t=1629899785; h=from:from:sender:sender:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references:list-id:list-help: list-unsubscribe:list-subscribe:list-post:dkim-signature; bh=Bw4b5v3/Mmjhp6+5JPLVZuZMEKpp3K5sW+sH90YG4fk=; b=mjZGcZl45NcRyAr36b8mKAxPLil4am9FtaU0x8SOPfX0ZEvUXe20tu20rATiuG313WQbgU 7XQTztJUAEVd3kLWjkLZyjRCTNtwjje6bsrtXi8GHPIbtq5UWQ3QWWS3WevyFyFzPl9fTj dkX5K5XJxgqNKS7KfgC6ciXI2kwRfAbT2/PtV6kmFzI/SIZIB4mU1nmMxsC/HcXp6y3gVq aQwMy5bwE9w5HlqYB3N4DPL1vq5zCp5roVDpH8fIaVosuR/5QvjPUjNLm+NoG48MtmhWNi xZ6XeeYRS8bKplkAB+tnjNaYS3+zyocRtzL8AUhslllRyI2M+YiFH/eEKl1p1w== ARC-Seal: i=1; s=key1; d=yhetil.org; t=1629899785; a=rsa-sha256; cv=none; b=idohsqEKU6XteYNIbqCCOjOHMzAAVwYXVECQuWHTpQxRv1Kx8W7zWxru99g5KGNj475NUd tLjyrVbploP0a/ZkP9CeMFa6vGQhIrEZlaOMmfdLNcuW658WAhCuw70oOVG+uU2mlpeoo1 XOhB3Ks5Km2vdxDLp1U/MmWY73wQeG+9HmTgnqAaQ+E0DoNgjNW9JYxChRGJPCW7alyUIf 44CLKI4n6geO5tLKVwOlKXqa1IZ8ObKPTeOu7xzosUtTO3bs4QwP6CqJUN7bdkNYKhUW9v LfrkACOtyPzxn//9VHnyoNeF8Y8Q/JAps4hqCG3u9BGgQt7mfvM6ZiK9qcM+zQ== ARC-Authentication-Results: i=1; aspmx1.migadu.com; dkim=pass header.d=arctype.co header.s=mail header.b=a4jjPriZ; dmarc=pass (policy=quarantine) header.from=arctype.co; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Spam-Score: -3.12 Authentication-Results: aspmx1.migadu.com; dkim=pass header.d=arctype.co header.s=mail header.b=a4jjPriZ; dmarc=pass (policy=quarantine) header.from=arctype.co; spf=pass (aspmx1.migadu.com: domain of guix-devel-bounces@gnu.org designates 209.51.188.17 as permitted sender) smtp.mailfrom=guix-devel-bounces@gnu.org X-Migadu-Queue-Id: 73A59265BD X-Spam-Score: -3.12 X-Migadu-Scanner: scn0.migadu.com X-TUID: fOIACsVads2y ------IFJP6XIDEHQZPO8J3Q6QTYMCO2DSAD Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello Sage, Why start yet another package manager from scratch? Are there any features= that couldn't just be implemented in Guix itself? I prefer development in = Racket over guile too, but Guile with it's low level system call/c compatib= ility plays to its strengths for the use cases involved in systems programm= ing a package manager=2E Did you discover guix after writing Xiden? I'm not understanding the use c= ases Xiden proposes that would be difficult to implement in Guix (for examp= le, GPG signature verification for package sources)=2E I don't think Racket= as a platform is compelling enough to justify the effort=2E You also compare Xiden to Racket as if it is a package manager to write ot= her package managers with=2E But you can also just call into Guix modules d= irectly and integrate guix into any other guile program and build a new pac= kage manager on top of it=2E I personally have used Racket to generate guil= e code to call guix modules to spawn processes for deterministic/generated = virtual machines=2E It is turtles all the way down=2E I could demo this in = private if you are interested=2E Anyways it sounds like there is some valuable research in Xiden, but I thi= nk it would be most beneficial at large to integrate some of the most usefu= l elements of your research into Guix, rather than try to reproduce the wor= k to bootstrap the world and also redefine or translate the thousands of pa= ckage definitions=2E The maintenance alone on the myriad package definition= s is enough to keep up with for all of us put together=2E My $0=2E02! Thanks for sharing=2E Sincerely, Ryan Sundberg -------- Original Message -------- From: Sage Gerard Sent: August 24, 2021 1:33:55 PM PDT To: guix-devel@gnu=2Eorg Subject: Xiden/Guix design comparison This is an offshoot Q&A based off Devos' questions in the "How did you han= dle making a GNU/Linux distribution?" thread=2E It pertains to how Guix com= pares to Xiden on some points=2E Given the length of both of our emails, I don't expect everyone to read th= is=2E Guix is mentioned for discussion purposes, but I use the -devel list = only because the questions did=2E Apologies to the mods if that's not an ex= cuse, since I'm not trying to distract from the purpose of the list=2E Read on only if you care=2E -- > Packages in Guix can specify which version of a package they use=2E [=2E= =2E=2E] Is something missing from Guix here? In Xiden, package versions and the notations used to express them are user= -defined=2E If you use semver and I use dates, we can still map our depende= ncy lists to the same package definitions=2E Xiden leaves it to us to agree= on a scheme for some set of packages=2E > SHA-384 and Keccak are separate things? [=2E=2E=2E] By definition, imple= mentations of SHA-384 have no reason to use Keccak=2E Correct=2E I'm just saying that Alice's decision to prefer a SHA-384 imple= mentation for using Keccak is not our business=2E Ditto for Bob and SHA-1= =2E Alice can force use of Keccak for software arriving on her system with = Xiden, without needing to discuss that with anyone=2E She's in charge of how a package manager works on her system, and we aren'= t=2E Xiden approaches CHFs similarly to OpenSSL in that implementations are= selected by providers, except in Xiden a knowledgeable user may assume the= role of a CHF implementation provider for their own system=2E I suspect we= 'd agree that is a terrible idea, which is why I use zero-trust defaults fo= r everything to mitigate risks=2E I allow for danger because most software distribution problems I've seen i= n the field boil down to smart people dealing with the lasting consequences= of outdated or rushed decisions=2E I'm not assuming competency, but I am a= ssuming that even those with bad practices still need to solve their distri= bution problems=2E >> unconditionally prefers cached installations, such that every defined a= rtifact >> has exactly one stored copy on disk=2E > > Coming from Guix, I don't know what this means=2E Does this mean only on= e version > of a package can be in the store at the same time, and building a newer = version > deletes the older version? How does this compare with =E2=80=98content d= eduplication=E2=80=99? Xiden has a cache for package inputs, and a cache for installed outputs=2E= The user has more control over the latter=2E Depending on the configuratio= n you use in a launcher, you can map all inputs to the same content-address= able filesystem, implying deduplication=2E However, you can also decide to = defeat the installed output cache for the sake of how SxS installations in = Xiden=2E That way you can store any number of any revision of any package i= n one store (Although I use the word "state" in the docs)=2E >> but only for artifacts from services authenticated by his operating sys= tem's certificates > > This is rather vague to me, unless you're referring to substitutes=2E > What does this mean for origin specifications? > > (source (origin > (method url-fetch) > (uri (string-append ["mirror://gnu/hello/hello-"](mirror://gnu/hello/hel= lo-) version > "=2Etar=2Egz")) > (sha256 ; imagine this was SHA-1 instead=2E > (base32 > "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i")))) > > Bob got the origin specification from Xiden (or something like a Guix ch= annel > but for Xiden)=2E How would this =E2=80=98artifact=E2=80=99 authenticate= d by =E2=80=98operating system > certificates=E2=80=99? I mean, when you let your operating system build = that origin > (via guix (or Xiden?), which you could consider part of the OS), it will= check > that hash, and it will be considered valid (=E2=80=98authenticated?=E2= =80=99) if the hash matches=2E > There are no certificates at play here=2E > > I suppose the OS (guix or Xiden?) could then =E2=80=98sign=E2=80=99 it, = but what value does that > provide to Bob? (Unless Bob runs a substitute server on their computer= =2E) Xiden distinguishes authenticating an artifact and authenticating a servic= e from which an artifact is specified=2E I mentioned the latter, so I was t= alking about SSL or TLS authentication=2E The word "source" means something different in Xiden=2E It refers to a str= ucture instance that implements a titular Racket generic=2E When "tapped", = a Xiden source yields an input port and an estimated number of bytes availa= ble for the port, or +inf=2E0 if no estimate is available=2E The user would= tell a launcher what budgets it would allow, if any, and Xiden will only r= ead bytes when the budget is in agreement with the estimate=2E When I say "artifact", I mean one of two things=2E - An (artifact) expression in a Xiden package input, which consists of a X= iden source, integrity information, and signature information (where availa= ble)=2E - Some blob of data that I can verify after a download=2E Definition #1 i= s just a more restricted form of the same idea=2E So to help us discuss this point I'd have translate your expression to a p= ackage input in Xiden as follows=2E I swapped the server and added a signat= ure expression to clarify something=2E (input "hello" (artifact (http-source (string-append ["https://packages=2Eexample=2Ecom/h= ello/"](https://packages=2Eexample=2Ecom/hello/) version "=2Etar=2Egz")) (integrity 'sha1 (base32 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89n= dq1i")) (signature (http-source ["https://packages=2Eexample=2Ecom/public-key=2Epe= m"](https://packages=2Eexample=2Ecom/public-key=2Epem)) (http-source (string-append ["https://packages=2Eexample=2Ecom/hello/"](ht= tps://packages=2Eexample=2Ecom/hello/) version "=2Etar=2Egz=2Esha1=2Esig"))= ))) I was saying that Bob can trust the CA that signed the certificates for pa= ckages=2Eexample=2Ecom, but handwaved over that because I was already writi= ng such a long email=2E The content of the =2Etar=2Egz file is authenticated separately=2E You can= see that I happened to use a PEM-encoded public key and a remote signature= file=2E The end-user's launcher would have to be compatible=2E If a user p= refers to use GPG and not allow for signatures that sit outside of a packag= e definition, their launcher would say so=2E There's an example in Xiden's source that shows how to swap out cryptograp= hic backends, so if you don't want to use OpenSSL or GPG, or if you want to= automate conversions of formats between the two, the option is available= =2E And if some of this sounds vague, that's because it is=2E Semantics of som= e forms are not fully defined without a launcher=2E e=2Eg=2E Xiden only und= erstands the (signature) form to authenticate data using assymetric cryprog= raphy, without caring about how exactly you'd do it=2E >> trusts digest creation and signature verification as implemented by his= host's dynamically--linked `libcrypto` library for use in the same process= =2E >> downloads content from GitHub packages > > AFAIK there is no such thing a =E2=80=98GitHub packages=E2=80=99=2E What= is a =E2=80=98GitHub package=E2=80=99, > precisely? I was referring to the output of the packages feature Microsoft rolled out= for GitHub=2E It integrates with the GitHub Actions feature for CI/CD=2E I am actually going to prepare an example where I use a GitHub repository = as a catalog of sorts, such that pushing to the repository adds a valid tar= get for a launcher=2E >> such that packages cannot conflict in a shared namespace > > =E2=80=98packages cannot conflict in a shared namespace=E2=80=99 is a bi= t vague=2E > Are you referring to =E2=80=98profile collisions=E2=80=99? About having = multiple versions > of the same package on the same system? Guix (and nix) can handle the la= tter=2E > When the =E2=80=98propagated-inputs=E2=80=99 are limited, installing mul= tiple packages each using > a different version of a package is possible=2E > > Is something missing from Guix here (please be very concrete)? [=2E=2E= =2E] I'm not sure what you mean here, can you illustrate this very concrete= ly? And is something missing in Guix here? I am not in a position to speak for others on what technical aspects are m= issing or inferior in Guix=2E That's not what I'm trying to do here=2E I only care about how easily a human being can switch between concrete det= ails in this problem domain, which means Xiden leaves some details for the = user to define=2E I'll give you concrete details on how I try to enable thi= s, and I'm not trying to say Guix is missing anything by doing so=2E One detail to note is that Xiden does not have profiles=2E With an unprivi= leged user, Xiden launchers all act like `ln -s = link-name`=2E I didn't think it needed to be more complicated than that=2E >> prefers Semantic Versions > > If upstream doesn't do =E2=80=98semver=E2=80=99, and instead uses a vers= ioning system like > used by TeX (3=2E14, 3=2E141, 3=2E1415, 3=2E14159, =2E=2E=2E), then Xide= n cannot somehow > let the package use =E2=80=98semver=E2=80=99 instead=2E Unless you want = to teach Xiden to > convince upstream to change their versioning system? A Xiden launcher may translate your preferred version notation to an upstr= eam scheme=2E >> These preferences are valid in Xiden's DSL for writing launchers=2E > > What is a launcher (compared to packages in Guix), and why would I want = one? The docs say what a launcher is=2E You want one if you want to adapt to social and technical differences betw= een package management systems=2E Xiden does not know what you want without= clarification from you, for similar reasons to why Vulkan ended up more ex= plicit than OpenGL=2E > Why shouldn't I just install packages and run them ("guix package -i tex= macs", > start texmacs from some application chooser menu of the desktop environm= ent)=2E Your question assumes `guix` is the command you want to run, which is fine= , but not everyone is you=2E I suspect that's why you are struggling to und= erstand how Xiden works, because Xiden does not prescribe itself as the sol= e option for how you put things on your system=2E It can also serve as a br= idge between your existing options=2E I think the pattern of everybody and their cat making a GNU/Linux distribu= tion with a new package manager raises questions about interoperability=2E = I'm trying to address that by saying that Xiden is to package managers what= Racket is to programming languages=2E The white paper covers this=2E >> The invariants for each launcher can differ as much as Cargo and Vortex= differ=2E >> What I wanted to do was allow users to declare their own invariants exp= licitly, >> but only when those invariants are wanted for subjective reasons=2E > > What are these invariants you're speaking of? I don't think you're refer= ring > to, say, translation invariance (=E2=80=98the laws of physics remain the= same under a > translation of the coordinate system=E2=80=99)=2E > > Or do you mean something like =E2=80=98Invariant (computer science), an = expression whose > value doesn't change during program execution=E2=80=99 ([](https://en=2Ewikipedia=2Eorg/wiki/Invariant= ))? https://stackoverflow=2Ecom/a/112088 > But what are the =E2=80=98expressions=E2=80=99 in this case? If I don't mean S-expressions, I am probably referring to a string of arbi= trary content with a meaning made up by the user=2E >> Xiden launchers have advantages over shell scripts in that we can both = still download >> and install dependencies from the same servers, and create reproducible= builds in terms >> of the same (bit-identical) state on disk=2E We just differ on details = that shouldn't impact >> how we work=2E It also helps us patch holes faster, since if (say) a ca= talog maintainer tells > > What is a =E2=80=98catalog maintainer=E2=80=99 (maybe it's like someone = with commit access to a =E2=80=98guix channel=E2=80=99?) A catch-all term I use for a person with a server that responds with artif= acts, by definition #2 of "artifact=2E" > Also, what =E2=80=98shell scripts=E2=80=99 are you referring to=2E Doesn't matter=2E That bit was me anticipating questions like "Why go this= abstract when I could just write a shell script?" >> us that a CHF was compromised, we can immediately revoke trust in the C= HF independently > > Maybe guix could gain a command "guix style transition-hash OLD NEW" to = automagically > rewrite package definitions to use the new hash algorithm instead of the= old=2E It will > need to download plenty of source code however, and it will entail a wor= ld-rebuild=2E Xiden doesn't have that problem=2E Or, more accurately, it doesn't take re= sponsibility for that problem=2E I think the real world is too messy to try= that=2E >> in our own clients=2E > > What are these =E2=80=98clients=E2=80=99 you're speaking of? I haven't b= een needing any =E2=80=98clients=E2=80=99 lately=2E In the context of Xiden, "client" may be a synonym for "launcher" if the l= auncher uses the network=2E >> The package definitions are all expressed in terms of what the launcher= s allow=2E > > If 'package definition' =3D (package definition as in Guix), then this i= s false, > Guix doesn't have a notion of =E2=80=98launchers=E2=80=99 and doesn't de= pend on =E2=80=98Xiden=E2=80=99=2E > What are you meaning, exactly? Also, I haven't found any package definit= ions in Xiden, > could you point me to any? The guide shows a command that prints where you can find examples=2E I hav= e spent over an hour typing before getting to this question, so I ask that = you please read the docs more=2E If you are looking for a definition for something like GNU coreutils, then= you won't find one without an associated launcher=2E I'll get to launcher = details in your other questions=2E I am not currently hosting any system-le= vel packages, so you won't get definitions from me=2E > Why would I want to =E2=80=98escape to a new distribution model=E2=80=99= ? Writing something like Guix > from scratch seems a lot of work to me, I would rather fork Guix (with a= few > likewise-minded collaborators) than to start over again=2E Or move to De= bian maybe=2E > I don't see what =E2=80=98Xiden=E2=80=99 could offer here=2E The details that would prompt one to fork a package manager source tree ca= n be trivially changed in a Xiden launcher=2E > I would rather not have to =E2=80=98adapt to problems=E2=80=99, I would = rather that upstream has something > that works (though I can understand if occasionally mistakes are made)= =2E We differ here=2E I need the ability to make my own decisions that may con= tradict upstream decisions at any time=2E That's a freedom I value and aim = to protect=2E -- I didn't bother with the rest of the email because I don't have time to re= peat answers found elsewhere, or define every other word=2E There are enoug= h contextual clues to help your own research=2E I'm happy to take further q= uestions, but please ask with the understanding that I've already been gene= rous with my time here=2E If there's a problem with Xiden's approach, pleas= e open a GitHub issue against zyrolasting/xiden=2E Thanks=2E ------IFJP6XIDEHQZPO8J3Q6QTYMCO2DSAD Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Hello Sage,

Why start yet another package manager from scra= tch? Are there any features that couldn't just be implemented in Guix itsel= f? I prefer development in Racket over guile too, but Guile with it's low l= evel system call/c compatibility plays to its strengths for the use cases i= nvolved in systems programming a package manager=2E

Did you discover= guix after writing Xiden? I'm not understanding the use cases Xiden propos= es that would be difficult to implement in Guix (for example, GPG signature= verification for package sources)=2E I don't think Racket as a platform is= compelling enough to justify the effort=2E

You also compare Xiden t= o Racket as if it is a package manager to write other package managers with= =2E But you can also just call into Guix modules directly and integrate gui= x into any other guile program and build a new package manager on top of it= =2E I personally have used Racket to generate guile code to call guix modul= es to spawn processes for deterministic/generated virtual machines=2E It is= turtles all the way down=2E I could demo this in private if you are intere= sted=2E

Anyways it sounds like there is some valuable research in Xi= den, but I think it would be most beneficial at large to integrate some of = the most useful elements of your research into Guix, rather than try to rep= roduce the work to bootstrap the world and also redefine or translate the t= housands of package definitions=2E The maintenance alone on the myriad pack= age definitions is enough to keep up with for all of us put together=2E
=
My $0=2E02! Thanks for sharing=2E

Sincerely,

Ryan Sundber= g

From: Sage Gerard <sage@sagegerard=2Ecom>
Sent: August 24, 2021 1:33:55 PM PDT
To: guix-devel@gnu=2Eorg
Subject: Xiden/Guix design comparison

This is an offshoot Q&A based off Devos' questions in the "How did you handle making a GNU/Linux distribution?" thread=2E It pertains to how Guix compares to Xiden on some points=2E

Given the length of both of our emails, I don't expect everyone to read this=2E Guix is mentioned for discussion purposes, but I use the -devel list only because the questions did=2E Apologies to the mods if that's not an excuse, since I'm not trying to distract from the purpose of the list=2E

Read on only if you care=2E

--

> Packages in Guix can specify which version of a package they use=2E [=2E=2E=2E] Is something missing from Guix here?

In Xiden, package versions and the notations used to express them are user-defined=2E If you use semver and I use dates, we can still map our dependency lists to the same package definitions=2E Xiden leaves it to us to agree on a scheme for some set of packages=2E

> SHA-384 and Keccak are separate things? [=2E=2E=2E] By definit= ion, implementations of SHA-384 have no reason to use Keccak=2E

Correct=2E I'm just saying that Alice's decision to prefer a SHA-384 implementation for using Keccak is not our business=2E Ditto for Bob and SHA-1=2E Alice can force use of Keccak for software arriving on her system with Xiden, without needing to discuss that with anyone=2E

She's in charge of how a package manager works on her system, and we aren't=2E Xiden approaches CHFs similarly to OpenSSL in that implementations are selected by providers, except in Xiden a knowledgeable user may assume the role of a CHF implementation provider for their own system=2E I suspect we'd agree that is a terrible idea, which is why I use zero-trust defaults for everything to mitigate risks=2E

I allow for danger because most software distribution problems I've seen in the field boil down to smart people dealing with the lasting consequences of outdated or rushed decisions=2E I'm not assuming competency, but I am assuming that even those with bad practices still need to solve their distribution problems=2E

unconditionally prefers cached installations, such that every defined artifact
has exactly one stored copy on disk=2E
Coming from Guix, I don't know what this means=2E Does this mean only one version
of a package can be in the store at the same time, and building a newer version
deletes the older version? How does this compare with =E2=80=98con= tent deduplication=E2=80=99?

Xiden has a cache for package inputs, and a cache for installed outputs=2E The user has more control over the latter=2E Depending on the configuration you use in a launcher, you can map all inputs to the same content-addressable filesystem, implying deduplication=2E However, you can also decide to defeat the installed output cache for the sake of how SxS installations in Xiden=2E That way you can store any number of any revision of any package in one store (Although I use the word "state" in the docs)=2E

but only for artifacts from services authenticated by his operating system's certificates
This is rather vague to me, unless you're referring to substitutes=2E
What does this mean for origin specifications?

(source (origin
(method url-fetch)
(uri (string-append "mirror://gnu/hello/hello-" version
"=2Etar=2Egz"))
(sha256 ; imagine this was SHA-1 instead=2E
(base32
"0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))))

Bob got the origin specification from Xiden (or something like a Guix channel
but for Xiden)=2E How would this =E2=80=98artifact=E2=80=99 authen= ticated by =E2=80=98operating system
certificates=E2=80=99? I mean, when you let your operating system = build that origin
(via guix (or Xiden?), which you could consider part of the OS), it will check
that hash, and it will be considered valid (=E2=80=98authenticated= ?=E2=80=99) if the hash matches=2E
There are no certificates at play here=2E

I suppose the OS (guix or Xiden?) could then =E2=80=98sign=E2=80= =99 it, but what value does that
provide to Bob? (Unless Bob runs a substitute server on their computer=2E)


Xiden distinguishes authenticating an artifact and authenticating a service from which an artifact is specified=2E I mentioned the latter, so I was talking about SSL or TLS authentication=2E

The word "source" means something different in Xiden=2E It refers to a structure instance that implements a titular Racket generic=2E When "tapped", a Xiden source yields an input port and an estimated number of bytes available for the port, or +inf=2E0 if no estimate is available=2E The user would tell a launcher what budgets it would allow, if any, and Xiden will only read bytes when the budget is in agreement with the estimate=2E

When I say "artifact", I mean one of two things=2E
  1. An (artifact) expression in a Xiden package input, which consists of a Xiden source, integrity information, and signature information (where available)=2E
  2. Some blob of data that I can verify after a download=2E Definition #1 is just a more restricted form of the same idea=2E
So to help us discuss this point I'd have translate your expression to a package input in Xiden as follows=2E I swapped the server and added a signature expression to clarify something=2E

(input "hello"
  (artifact (http-source (string-append "https://packages=2Eexample=2Ecom/hello/" version "= =2Etar=2Egz"))
 &n= bsp;          (integrity 'sha1 (base32 "0ssi1wpaf7plaswqqjwigppsg5fyh99vdlb9kzl7c9lng89ndq1i"))
 &n= bsp;          (signature (http-source "https://packages=2Eexample=2Ecom/public-key=2E= pem")
 &n= bsp;            = ;         (http-source (string-append "https://packages=2Eexample=2Ecom/hello/" version "=2Etar=2Egz=2Esha1=2Esig")))))

I was saying that Bob can trust the CA that signed the certificates for packages=2Eexample=2Ecom, bu= t handwaved over that because I was already writing such a long email=2E

The content of the =2Etar=2Egz fi= le is authenticated separately=2E You can see that I happened to use a PEM-encoded public key and a remote signature file=2E The end-user's launcher would have to be compatible=2E If a user prefers to use GPG and not allow for signatures that sit outside of a package definition, their launcher would say so=2E

There's an example in Xiden's source that shows how to swap out cryptographic backends, so if you don't want to use OpenSSL or GPG, or if you want to automate conversions of formats between the two, the option is available=2E

And if some of this sounds vague, that's because it is=2E Semantics of some forms are not fully defined without a launcher=2E e=2Eg=2E Xiden only understands the (signature) form to authenticate data using assymetric cryprography, without caring about how exactly you'd do it=2E

trusts digest creation and signature verification as implemented by his host's dynamically--linked `libcrypto` library for use in the same process=2E
downloads content from GitHub packages
AFAIK there is no such thing a =E2=80=98GitHub packages=E2=80=99= =2E What is a =E2=80=98GitHub package=E2=80=99,
precisely?
I was referring to the output of the packages feature Microsoft rolled out for GitHub=2E It integrates with the GitHub Actions feature for CI/CD=2E

I am actually going to prepare an example where I use a GitHub repository as a catalog of sorts, such that pushing to the repository adds a valid target for a launcher=2E

such that packages cannot conflict in a shared namespace
=E2=80=98packages cannot conflict in a shared namespace=E2=80=99 i= s a bit vague=2E
Are you referring to =E2=80=98profile collisions=E2=80=99? About h= aving multiple versions
of the same package on the same system? Guix (and nix) can handle the latter=2E
When the =E2=80=98propagated-inputs=E2=80=99 are limited, installi= ng multiple packages each using
a different version of a package is possible=2E

Is something missing from Guix here (please be very concrete)? [=2E=2E=2E] I'm not sure what you mean here, can you illustrate th= is very concretely? And is something missing in Guix here?

I am not in a position to speak for others on what technical aspects are missing or inferior in Guix=2E That's not what I'm trying to do here=2E

I only care about how easily a human being can switch between concrete details in this problem domain, which means Xiden leaves some details for the user to define=2E I'll give you concrete details on how I try to enabl= e this, and I'm not trying to say Guix is missing anything by doing so=2E

One detail to note is that Xiden does not have profiles=2E With an unprivileged user, Xiden launchers all act like `ln -s <totally-made-up-notation> link-name`=2E I didn't think it needed to be more complicated than that=2E

prefers Semantic Versions
If upstream doesn't do =E2=80=98semver=E2=80=99, and instead uses = a versioning system like
used by TeX (3=2E14, 3=2E141, 3=2E1415, 3=2E14159, =2E=2E=2E), the= n Xiden cannot somehow
let the package use =E2=80=98semver=E2=80=99 instead=2E Unless you= want to teach Xiden to
convince upstream to change their versioning system?

A Xiden launcher may translate your preferred version notation to an upstream scheme=2E
These preferences are valid in Xiden's DSL for writing launchers=2E
What is a launcher (compared to packages in Guix), and why would I want one?

The docs say what a launcher is=2E

You want one if you want to adapt to social and technical differences between package management systems=2E Xiden does not know what you want without clarification from you, for similar reasons to why Vulkan ended up more explicit than OpenGL=2E

Why shouldn't I just install packages and run them ("guix package -i texmacs",
start texmacs from some application chooser menu of the desktop environment)=2E

Your question assumes `guix` is the command you want to run, which is fine, but not everyone is you=2E I suspect that's why you are struggling to understand how Xiden works, because Xiden does not prescribe itself as the sole option for how you put things on your system=2E It can also serve as a bridge between your existing options=2E

I think the pattern of everybody and their cat making a GNU/Linux distribution with a new package manager raises questions about interoperability=2E I'm trying to address that by saying that Xiden is to package managers what Racket is to programming languages=2E The white paper covers this=2E

The invariants for each launcher can differ as much as Cargo and Vortex differ=2E
What I wanted to do was allow users to declare their own invariants explicitly,
but only when those invariants are wanted for subjective reasons=2E
What are these invariants you're speaking of? I don't think you're referring
to, say, translation invariance (=E2=80=98the laws of physics rema= in the same under a
translation of the coordinate system=E2=80=99)=2E

Or do you mean something like =E2=80=98Invariant (computer science= ), an expression whose
value doesn't change during program execution=E2=80=99 (<https://en=2Ewikipedia=2Eorg/wiki/Invariant><= /a>)?
https://stackoverflow=2Ecom= /a/112088

But what are the =E2=80=98expressions=E2= =80=99 in this case?
If I don't mean S-expressions, I am probably referring to a string of arbitrary content with a meaning made up by the user=2E
Xiden launchers have advantages over shell scripts in that we can both still download
and install dependencies from the same servers, and create reproducible builds in terms
of the same (bit-identical) state on disk=2E We just differ on details that shouldn't impact
how we work=2E It also helps us patch holes faster, since if (say) a catalog maintainer tells
What is a =E2=80=98catalog maintainer=E2=80=99 (maybe it's like so= meone with commit access to a =E2=80=98guix channel=E2=80=99?)
A catch-all term I use for a person with a server that responds with artifacts, by definition #2 of "artifact=2E"

Also, what =E2=80=98shell scripts=E2=80=99= are you referring to=2E

Doesn't matter=2E That bit was me anticipating questions like "Why go this abstract when I could just write a shell script?"

us that a CHF was compromised, we can immediately revoke trust in the CHF independently
Maybe guix could gain a command "guix style transition-hash OLD NEW" to automagically
rewrite package definitions to use the new hash algorithm instead of the old=2E It will
need to download plenty of source code however, and it will entail a world-rebuild=2E

Xiden doesn't have that problem= =2E Or, more accurately, it doesn't take responsibility for that problem=2E I think the real world is too messy to try that=2E

in our own clients=2E
What are these =E2=80=98clients=E2=80=99 you're speaking of? I hav= en't been needing any =E2=80=98clients=E2=80=99 lately=2E

In the context of Xiden, "client" may be a synonym for "launcher" if the launcher uses the network=2E<= br>

The package definitions are all expressed in terms of what the launchers allow=2E
If 'package definition' =3D (package definition as in Guix), then this is false,
Guix doesn't have a notion of =E2=80=98launchers=E2=80=99 and does= n't depend on =E2=80=98Xiden=E2=80=99=2E
What are you meaning, exactly? Also, I haven't found any package definitions in Xiden,
could you point me to any?

The guide shows a command that prints where you can find examples=2E I have spent over an hour typing before getting to this question, so I ask that you please read the docs more=2E

If you are looking for a definition for something like GNU coreutils, then you won't find one without an associated launcher=2E I'll get to launcher details in your other questions=2E I am not currently hosting any system-level packages, so you won't get definitions from me=2E

Why would I want to =E2=80=98escape to a n= ew distribution model=E2=80=99? Writing something like Guix
from scratch seems a lot of work to me, I would rather fork Guix (with a few
likewise-minded collaborators) than to start over again=2E Or move to Debian maybe=2E
I don't see what =E2=80=98Xiden=E2=80=99 could offer here=2E
The details that would prompt one to fork a package manager source tree can be trivially changed in a Xiden launcher=2E

I would rather not have to =E2=80=98adapt = to problems=E2=80=99, I would rather that upstream has something
that works (though I can understand if occasionally mistakes are made)=2E
We differ here=2E I need the ability to make my own decisions that may contradict upstream decisions at any time=2E That's a freedom I value and aim to protect=2E

--

I didn't bother with the rest of the email because I don't have time to repeat answers found elsewhere, or define every other word=2E There are enough contextual clues to help your own research=2E I'm happy to take further questions, but please ask with the understanding that I've already been generous with my time here=2E If there's a problem with Xiden's approach, please open a GitHub issue against zyrolasting/xiden=2E

Thanks=2E
------IFJP6XIDEHQZPO8J3Q6QTYMCO2DSAD--