Public guix offload server

Public guix offload server

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 1149 bytes --]


Hi Guix,

This is a follow-up to the "Incentives for review" thread.

I think it would be nice to have a public/semi-public access `guix
offload' server. My own machine is relatively slow and has a spinning
disk. I can't really review "big patches" (in the sense of build time)
without ruining a few days of my life. So, I usually refrain from
reviewing patches to packages with a lot of dependents. And, I don't
participate in world-rebuilding tasks such as merging core-updates. This
problem is becoming worse as Guix grows larger and the dependencies
become more complex. Perhaps, if we had a public `guix offload' server
to which I could offload cumbersome builds, I might feel more encouraged
to review tough patches.

If security is a problem with a public access guix offload server, we
could make it semi-public and available at least to people with commit
access. Currently, guix offload requires mutual trust between the master
and the build machines. If we could make the trust only one-way,
security might be less of an issue.

WDYT? How does everyone else handle big builds? Do you have access to
powerful workstations?

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Public guix offload server

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1855 bytes --]

Hi Arun,

Arun Isaac 写道：
> If security is a problem with a public access guix offload 
> server, we
> could make it semi-public and available at least to people with 
> commit
> access.

Giving access only to people with commit access is a given, but 
any shared offload server is a huge shared security risk.

Guix is not content-addressed.  Any [compromised] user can upload 
arbitrary malicious binaries with store hashes identical to the 
legitimate build.  These malicious binaries can then be downloaded 
by other clients, which presumably all have commit access.

Now the attacker almost certainly has covert access to one or more 
user accounts that can push signed commits to Guix upstream.

> Currently, guix offload requires mutual trust between the master
> and the build machines. If we could make the trust only one-way,
> security might be less of an issue.

It might!  It's easy to imagine a second, less powerful offload 
protocol where clients can submit only derivations to be built by 
the remote daemon, plus fixed-output derivations.  None of the 
‘let me send the entire binary toolchain so you don't have to 
build it from scratch’ of the current protocol.  This at least 
removes their control over the source hash.

At that point, one might consider dropping SSH account-based 
access in favour of a minimal job submission API, and just return 
the results through guix publish or so…?  OTOH, that's yet another 
code path.

> WDYT? How does everyone else handle big builds? Do you have 
> access to
> powerful workstations?

By waiting, and planning.  I'm lucky to have a ridiculously 
overpowered ThinkPad for its age and a newer headless tower at 
home that can run builds 24/7, but nothing close to a ‘powerful 
workstation’ by industry standards.

Zzz,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Public guix offload server

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 927 bytes --]

On Wed, Oct 20, 2021 at 11:06:05PM +0200, Tobias Geerinckx-Rice wrote:
> Guix is not content-addressed.  Any [compromised] user can upload arbitrary
> malicious binaries with store hashes identical to the legitimate build.
> These malicious binaries can then be downloaded by other clients, which
> presumably all have commit access.

Interesting... I'm not at all familiar with how `guix offload` works,
because I've never used it. But it's surprising to me that this would be
possible. Although after one minute of thought, I'm not sure why it
wouldn't be.

However, the Guix security model trusts committers implicitly. So, if
the committers' shared offload server had proper access control, one
might consider it "good enough" in terms of security. Although the
possibility of spreading malicious binaries is much scarier than what
could be achieved by committing to guix.git, because of the relative
lack of transparency.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Public guix offload server

[Leo Famulari]

[-- Attachment #1: Type: text/plain, Size: 541 bytes --]

On Thu, Oct 21, 2021 at 02:23:49AM +0530, Arun Isaac wrote:
> WDYT? How does everyone else handle big builds? Do you have access to
> powerful workstations?

For my first several years with Guix... I handled big builds patience
and care.

I could have spent a small amount of money on powerful yet ephemeral
cloud computing.

Now I have access to a very powerful system on which I can test builds.

I agree that the Guix project should offer some powerful compute
resources to Guix developers. Efficiency is important for staying
motivated.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: Public guix offload server

[zimoun]

Hi Tobias,

On Wed, 20 Oct 2021 at 23:06, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> Giving access only to people with commit access is a given, but 
> any shared offload server is a huge shared security risk.
>
> Guix is not content-addressed.  Any [compromised] user can upload 
> arbitrary malicious binaries with store hashes identical to the 
> legitimate build.  These malicious binaries can then be downloaded 
> by other clients, which presumably all have commit access.
>
> Now the attacker almost certainly has covert access to one or more 
> user accounts that can push signed commits to Guix upstream.

If I understand correctly, if a committer offloads to say Berlin or
Bayfront, your concern is that the output will be in the publicly
exposed store.  Right?

If yes, one could imagine two stores: one populated by CI as it is
currently done and another one mounted elsewhere considered as sandbox
and regularly garbage collected.

For instance, one could imagine a dedicated VM for all the committers
who require some CPU power.

I mean, it is some system administration work, but is it not technically
feasible?


> At that point, one might consider dropping SSH account-based 
> access in favour of a minimal job submission API, and just return 
> the results through guix publish or so…?  OTOH, that's yet another 
> code path.

A minimal job submission API with token would be ideal, IMHO.  But it
falls into:

        Now is better than never.
        Although never is often better than *right* now.

                                    – python -c 'import this' –


> By waiting, and planning.  I'm lucky to have a ridiculously 
> overpowered ThinkPad for its age and a newer headless tower at 
> home that can run builds 24/7, but nothing close to a ‘powerful 
> workstation’ by industry standards.

I sympathize with Arun’s requests.  For instance, it is impossible to
review Julia packages using my old laptop.  Even, it takes ages to just
compile Guix from sources and it is becoming worse and worse.
Hopefully, I am lucky and I have remote access to some workstations at
work.

Yes, we can wait and plan for a better solution for helping committers
to do their review. ;-)


Cheers,
simon

Re: Public guix offload server

[Joshua Branson]

Leo Famulari <leo@famulari.name> writes:

> On Thu, Oct 21, 2021 at 02:23:49AM +0530, Arun Isaac wrote:
>> WDYT? How does everyone else handle big builds? Do you have access to
>> powerful workstations?
>
> Now I have access to a very powerful system on which I can test builds.
>
> I agree that the Guix project should offer some powerful compute
> resources to Guix developers. Efficiency is important for staying
> motivated.

I've got an old Dell Optiplex 7020 with 30 gigs of RAM with a 3TB
hard-drive just sitting around.  My landlord and ISP is ok with me
running a server.  I just set everything up.  Would this be
powerful/interesting to some?

-- 
Joshua Branson (jab in #guix)
Sent from Emacs and Gnus
  https://gnucode.me
  https://video.hardlimit.com/accounts/joshua_branson/video-channels
  https://propernaming.org
  "You can have whatever you want, as long as you help
enough other people get what they want." - Zig Ziglar
  

Re: Public guix offload server

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1176 bytes --]

Hi Simon,

zimoun 写道：
> If I understand correctly, if a committer offloads to say Berlin 
> or
> Bayfront, your concern is that the output will be in the 
> publicly
> exposed store.  Right?

No, that would be far worse.  I'm considering only a ‘private’ 
offload server shared by several trusted users, where one 
compromised (whether technically or mentally :-) user can easily 
‘infect’ other contributors in a way that's very hard to detect. 
‘Trusting trust’ comes to mind.

> For instance, one could imagine a dedicated VM for all the 
> committers
> who require some CPU power.

Right, that's what I'm describing in my previous mail.

Now, we could spin up a separate VM for each user, and just take 
the efficiency hit…  Users would be safe from anything but 
VM-escape exploits (which exist but are rare).

> A minimal job submission API with token would be ideal, IMHO. 
> But it
> falls into:
>
>         Now is better than never.
>         Although never is often better than *right* now.
>
>                                     – python -c 'import this' –

What does this mean?

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Public guix offload server

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 400 bytes --]

Joshua Branson 写道：
> I've got an old Dell Optiplex 7020 with 30 gigs of RAM with a 
> 3TB
> hard-drive just sitting around.  My landlord and ISP is ok with 
> me
> running a server.  I just set everything up.  Would this be
> powerful/interesting to some?

Well, not going to lie: yes.

I've heard that US power is relatively cheap, but are you sure? 
:-)

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Public guix offload server

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 1647 bytes --]

Leo,

Leo Famulari 写道：
> Interesting... I'm not at all familiar with how `guix offload` 
> works,
> because I've never used it. But it's surprising to me that this 
> would be
> possible. Although after one minute of thought, I'm not sure why 
> it
> wouldn't be.

Very quickly:

- You send an offload request to the offload server, but you also 
  get so send any remotely missing store items that you already 
  have, as opaque binaries (icecat could be tetris instead).

This is why the offload server has to trust your key.  It's 
valuable and shouldn't be removed, but making it optional[0] 
shouldn't be ‘too hard’.

- The offload sends back one or more store items, which is why you 
  trust it.  This part is just substitution in a different form 
  (SSH vs. HTTPS etc.)

> However, the Guix security model trusts committers 
> implicitly. So, if
> the committers' shared offload server had proper access control, 
> one
> might consider it "good enough" in terms of security.

The two are *SO* different as to be incomparable IMO.

You do point out the difference, so I guess we just assess it very 
differently:

> Although the
> possibility of spreading malicious binaries is much scarier than 
> what
> could be achieved by committing to guix.git, because of the 
> relative
> lack of transparency.

Gotta run,

T G-R

[0]: Which would create the

  “second, less powerful offload protocol where clients can submit
   only derivations to be built by the remote daemon, plus
   fixed-output derivations”

I imagined.  But this is still hand-waving at this point.  :-)

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Public guix offload server

[Vagrant Cascadian]

[-- Attachment #1: Type: text/plain, Size: 1427 bytes --]

On 2021-10-21, Joshua Branson wrote:
> Leo Famulari <leo@famulari.name> writes:
>
>> On Thu, Oct 21, 2021 at 02:23:49AM +0530, Arun Isaac wrote:
>>> WDYT? How does everyone else handle big builds? Do you have access to
>>> powerful workstations?
>>
>> Now I have access to a very powerful system on which I can test builds.
>>
>> I agree that the Guix project should offer some powerful compute
>> resources to Guix developers. Efficiency is important for staying
>> motivated.
>
> I've got an old Dell Optiplex 7020 with 30 gigs of RAM with a 3TB
> hard-drive just sitting around.  My landlord and ISP is ok with me
> running a server.  I just set everything up.  Would this be
> powerful/interesting to some?

I wonder if OSUOSL (or maybe other organizations) would be willing to
provide a nice big server with access restricted to guix committers or
something?

  https://osuosl.org/services/hosting/

I know they provide some very capable machines for reproducible builds
and numerous other projects... (machines capable of running many virtual
machines, FWIW)

(And some POWER9 machines for guix, if I recall...)

Then it could run "guix publish" and people could log in and run builds
as they wished, and download the results via typical methods... still a
little danger of poisoning the store if people use the binaries on
production hardware... but... can only do so much hand-holding, right?
:)


live well,
  vagrant

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: Public guix offload server

[zimoun]

Hi Tobias,

On Thu, 21 Oct 2021 at 18:31, Tobias Geerinckx-Rice <me@tobias.gr> wrote:
> zimoun 写道：

>> If I understand correctly, if a committer offloads to say Berlin 
>> or
>> Bayfront, your concern is that the output will be in the 
>> publicly
>> exposed store.  Right?
>
> No, that would be far worse.  I'm considering only a ‘private’ 
> offload server shared by several trusted users, where one 
> compromised (whether technically or mentally :-) user can easily 
> ‘infect’ other contributors in a way that's very hard to detect. 
> ‘Trusting trust’ comes to mind.

Thanks for explaining.

Unseriously, I do not the see the difference when several trusted users
push to a Git repo, where one compromised user can easily ’infect’ other
contributors in a way that’s very hard to detect. ;-)

If a compromised user offloads something, how other users of the same
server would get this compromised stuff?  Maybe I miss something.

Considering trusted users (i.e., not conscientiously malicious), the
surface of the attack is reproducible builds; similarly to the current
situation of substitutes by CI.  What do I miss?

Well, I do not see the difference between a remote offload server and a
shared store on cluster (although probably worse because many users – at
least some of I know – of clusters often do not really understand what
they do when using Guix ;-)).


> Now, we could spin up a separate VM for each user, and just take 
> the efficiency hit…  Users would be safe from anything but 
> VM-escape exploits (which exist but are rare).

Do you mean that trusted users would try WM-escape exploits?


>> A minimal job submission API with token would be ideal, IMHO. 
>> But it
>> falls into:
>>
>>         Now is better than never.
>>         Although never is often better than *right* now.
>>
>>                                     – python -c 'import this' –
>
> What does this mean?

It is The Zen of Python. :-) These sentences express the complexity of
the right balance, IMHO.  Sorry if it was unclear.  Otherwise, the
complete Zen reads:

--8<---------------cut here---------------start------------->8---
$ python -c 'import this'
The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!
--8<---------------cut here---------------end--------------->8---


Cheers,
simon

Re: Public guix offload server

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 1007 bytes --]


Hi,

>> Currently, guix offload requires mutual trust between the master and
>> the build machines. If we could make the trust only one-way, security
>> might be less of an issue.
>
> It might!  It's easy to imagine a second, less powerful offload
> protocol where clients can submit only derivations to be built by the
> remote daemon, plus fixed-output derivations.  None of the ‘let me
> send the entire binary toolchain so you don't have to build it from
> scratch’ of the current protocol.  This at least removes their control
> over the source hash.

I just realized we might already have something close to this second,
less powerful offload protocol that needs only one-way trust. According
to the NEWS file, since Guix 0.13.0, the GUIX_DAEMON_SOCKET environment
variable lets us specify remote daemons. See "(guix) The Store" in the
manual for detailed documentation. The only thing missing is some way to
retrieve the built output from the remote store.

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Public guix offload server

[Jonathan McHugh]

October 21, 2021 8:10 PM, "zimoun" <zimon.toutoune@gmail.com> wrote:

> 
>> Now, we could spin up a separate VM for each user, and just take
>> the efficiency hit… Users would be safe from anything but
>> VM-escape exploits (which exist but are rare).
> 
> Do you mean that trusted users would try WM-escape exploits?
> 

The world has been formed by warewolves inside communities purposely causing harm. Looking further back, Oliver the Spy is a classic examplar of trust networks being hollowed out.


Jonathan

Re: Public guix offload server

[zimoun]

Hi,

On Thu, 21 Oct 2021 at 21:15, "Jonathan McHugh" <indieterminacy@libre.brussels> wrote:
> October 21, 2021 8:10 PM, "zimoun" <zimon.toutoune@gmail.com> wrote:

>>> Now, we could spin up a separate VM for each user, and just take
>>> the efficiency hit… Users would be safe from anything but
>>> VM-escape exploits (which exist but are rare).
>> 
>> Do you mean that trusted users would try WM-escape exploits?
>
> The world has been formed by warewolves inside communities purposely
> causing harm. Looking further back, Oliver the Spy is a classic
> examplar of trust networks being hollowed out.

I cannot assume that on one hand one trusted person pushes to the main
Git repo in good faith and on other hand this very same trusted person
behaves as a warewolves using a shared resource.

For sure, one can always abuse the trust.  Based on this principle, we
could stop any collaborative work right now.  The real question is the
evaluation of the risk of such abuse by trusted people after long period
of collaboration (that’s what committer usually means).

Various examples exist on this kind of abused trust.  Oliver the Spy is
one, Mark Kennedy/Stone is another recent one.

Anyway! :-)

All the best,
simon

Re: Public guix offload server

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 2260 bytes --]

All,

zimoun 写道：
>>> Do you mean that trusted users would try WM-escape exploits?
>> The world has been formed by warewolves inside communities 
>> purposely
>> causing harm. Looking further back, Oliver the Spy is a classic
>> examplar of trust networks being hollowed out.

So…

> I cannot assume that on one hand one trusted person pushes to 
> the main
> Git repo in good faith and on other hand this very same trusted 
> person
> behaves as a warewolves using a shared resource.

…li'l' sleepy here, bewarned, but before this gets out of hand: I 
never implied direct abuse of trust by committers.  I don't 
consider it the main threat[0].

There are the people you meet at FOSDEM and the users who log into 
machines.  Both can be compromised, but the latter are much easier 
and more likely to be.

Such compromise is not laughable or hypothetical: it happens 
*constantly*.  It's how kernel.org was utterly owned[1].

Trusting people not to be evil is not the same as having to trust 
the opsec habits of every single one of them.  Trust isn't 
transitive.  Personally, I don't think a rogue zimoun will 
suddenly decide to abuse us.  I think rogues will abuse zimoun the 
very first chance they get.

That's not a matter of degree: it's a whole different threat 
model, as is injecting arbitrary binaries vs. pushing malicious 
code commits.  Both are bad news, but there's an order of 
magnitude difference between the two.

> For sure, one can always abuse the trust.  Based on this 
> principle, we
> could stop any collaborative work right now.  The real question 
> is the
> evaluation of the risk of such abuse by trusted people after 
> long period
> of collaboration (that’s what committer usually means).

Isn't that the kind of hands-up-in-the-air why-bother 
nothing's-perfect fatalism I thought your Python quote (thanks!) 
was trying to warn me about?  ;-)

Zzz,

T G-R

[0]: That's probably no more than an optimistic human flaw on my 
part and ‘disgruntled ex-whatevers’ are probably more of a threat 
that most orgs dare to admit.

[1]: I know, ancient history, but I'm working from memory here. 
I'm sure it would be trivial to find a more topical example.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]

Re: Public guix offload server

[jbranso]

October 21, 2021 12:44 PM, "Tobias Geerinckx-Rice" <me@tobias.gr> wrote:

> Joshua Branson 写道：
> 
>> I've got an old Dell Optiplex 7020 with 30 gigs of RAM with a
>> 3TB
>> hard-drive just sitting around. My landlord and ISP is ok with
>> me
>> running a server. I just set everything up. Would this be
>> powerful/interesting to some?
> 
> Well, not going to lie: yes.
> 
> I've heard that US power is relatively cheap, but are you sure?
> :-)
 
I guess we'll find out how expensive it gets...if I can't get enough 
donations to keep it running, I could always shut it off.

> Kind regards,
> 
> T G-R

Re: Public guix offload server

[Jonathan McHugh]

I have utmost confidence in the Guix project, it has lots of smart and inquisitive people to suppliment its accountable structures - a very useful bulwark against exploitative behaviour!

====================
Jonathan McHugh
indieterminacy@libre.brussels

October 22, 2021 12:59 AM, "Tobias Geerinckx-Rice" <me@tobias.gr> wrote:

> All,
> 
> zimoun 写道：
> 
>> Do you mean that trusted users would try WM-escape exploits?
>>> The world has been formed by warewolves inside communities
>>> purposely
>>> causing harm. Looking further back, Oliver the Spy is a classic
>>> examplar of trust networks being hollowed out.
> 
> So…
> 
>> I cannot assume that on one hand one trusted person pushes to
>> the main
>> Git repo in good faith and on other hand this very same trusted
>> person
>> behaves as a warewolves using a shared resource.
> 
> …li'l' sleepy here, bewarned, but before this gets out of hand: I
> never implied direct abuse of trust by committers. I don't
> consider it the main threat[0].
> 
> There are the people you meet at FOSDEM and the users who log into
> machines. Both can be compromised, but the latter are much easier
> and more likely to be.
> 
> Such compromise is not laughable or hypothetical: it happens
> *constantly*. It's how kernel.org was utterly owned[1].
> 
> Trusting people not to be evil is not the same as having to trust
> the opsec habits of every single one of them. Trust isn't
> transitive. Personally, I don't think a rogue zimoun will
> suddenly decide to abuse us. I think rogues will abuse zimoun the
> very first chance they get.
> 
> That's not a matter of degree: it's a whole different threat
> model, as is injecting arbitrary binaries vs. pushing malicious
> code commits. Both are bad news, but there's an order of
> magnitude difference between the two.
> 
>> For sure, one can always abuse the trust. Based on this
>> principle, we
>> could stop any collaborative work right now. The real question
>> is the
>> evaluation of the risk of such abuse by trusted people after
>> long period
>> of collaboration (that’s what committer usually means).
> 
> Isn't that the kind of hands-up-in-the-air why-bother
> nothing's-perfect fatalism I thought your Python quote (thanks!)
> was trying to warn me about? ;-)
> 
> Zzz,
> 
> T G-R
> 
> [0]: That's probably no more than an optimistic human flaw on my
> part and ‘disgruntled ex-whatevers’ are probably more of a threat
> that most orgs dare to admit.
> 
> [1]: I know, ancient history, but I'm working from memory here.
> I'm sure it would be trivial to find a more topical example.

Re: Public guix offload server

[zimoun]

Hi Tobias,

I understand your point of view.

On Fri, 22 Oct 2021 at 00:16, Tobias Geerinckx-Rice <me@tobias.gr> wrote:

> Trusting people not to be evil is not the same as having to trust 
> the opsec habits of every single one of them.  Trust isn't 
> transitive.  Personally, I don't think a rogue zimoun will 
> suddenly decide to abuse us.  I think rogues will abuse zimoun the 
> very first chance they get.

From my understanding, here is the net of our “disagreement”.

> That's not a matter of degree: it's a whole different threat 
> model, as is injecting arbitrary binaries vs. pushing malicious 
> code commits.  Both are bad news, but there's an order of 
> magnitude difference between the two.

And I miss the threat model about “injecting binaries” in the case of
shared offload.  Anyway. :-)

Let move forward and discuss another solution than the usual offload.

You pointed the idea «one might consider dropping SSH account-based
access in favour of a minimal job submission API, and just return the
results through guix publish or so…?  OTOH, that's yet another code
path.»

Imagine another Cuirass instance where any committer could add [1] their
own branch.  It would act as this minimal job submission API.

1: <https://ci.guix.gnu.org/specification/add/>

The questions are the authentication to this Cuirass instance and how
Cuirass deals with rebased branch (which would happen).

WDYT?


Cheers,
simon

Re: Public guix offload server

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 870 bytes --]


Hi zimoun,

> Imagine another Cuirass instance where any committer could add [1] their
> own branch.  It would act as this minimal job submission API.
>
> 1: <https://ci.guix.gnu.org/specification/add/>
>
> The questions are the authentication to this Cuirass instance and how
> Cuirass deals with rebased branch (which would happen).

I don't think we need Cuirass. We could just use the remote guix-daemon
features that are already in Guix.

$ export GUIX_DAEMON_SOCKET=ssh://charlie@sandbox.guix.gnu.org:22
$ guix build foo

Then we just need to copy over the build outputs from the remote
host. Maybe, `guix copy' could be used.

With this method, we already have all the software that we
need. Authentication is handled via ssh. And, there is no need for trust
between users of the build server. Now, somebody just needs to set up
the hardware! :-)

Regards,
Arun

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Public guix offload server

[zimoun]

Hi Arun,

On Sat, 23 Oct 2021 at 11:19, Arun Isaac <arunisaac@systemreboot.net> wrote:

>> Imagine another Cuirass instance where any committer could add [1] their
>> own branch.  It would act as this minimal job submission API.

> I don't think we need Cuirass. We could just use the remote guix-daemon
> features that are already in Guix.

This is exactly what I have in mind.  Especially, we asked the
difference between a remote offload server and a shared store on
cluster. :-)

From my understanding, Tobias’s point is that…

> $ export GUIX_DAEMON_SOCKET=ssh://charlie@sandbox.guix.gnu.org:22
> $ guix build foo

…requires an SSH access by ’charlie’ to sandbox.guix.gnu.org,  And they
think this access is risky.


Cheers,
simon

Re: Public guix offload server

[Arun Isaac]

[-- Attachment #1: Type: text/plain, Size: 387 bytes --]


>> $ export GUIX_DAEMON_SOCKET=ssh://charlie@sandbox.guix.gnu.org:22
>> $ guix build foo
>
> …requires an SSH access by ’charlie’ to sandbox.guix.gnu.org,  And they
> think this access is risky.

We could provide SSH access but no shell access. We could use some
restricted shell in the spirit of git-shell. All we need is port
forwarding through SSH, not shell access.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 524 bytes --]

Re: Public guix offload server

[indieterminacy]

Hi Arun,

Researching git-shell, I noticed an example of how Less could be
exploited to increase. privileges:
=> https://hackaday.com/2017/05/10/git-shell-bypass-less-is-more/

It suggests enabling the no-pty flag to mitigate this.

I think it would be great to utilise git-shell (and I am interested in
it for my own activities.

Do anybody have detailed research regarding SSH access with no shell access?

Kind regards,

Jonathan

Arun Isaac <arunisaac@systemreboot.net> writes:

> [[PGP Signed Part:Undecided]]
>
>>> $ export GUIX_DAEMON_SOCKET=ssh://charlie@sandbox.guix.gnu.org:22
>>> $ guix build foo
>>
>> …requires an SSH access by ’charlie’ to sandbox.guix.gnu.org,  And they
>> think this access is risky.
>
> We could provide SSH access but no shell access. We could use some
> restricted shell in the spirit of git-shell. All we need is port
> forwarding through SSH, not shell access.
>
> [[End of PGP Signed Part]]

Re: Public guix offload server

[Ludovic Courtès]

Hi,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

> Arun Isaac 写道：

[...]

>> Currently, guix offload requires mutual trust between the master
>> and the build machines. If we could make the trust only one-way,
>> security might be less of an issue.
>
> It might!  It's easy to imagine a second, less powerful offload
> protocol where clients can submit only derivations to be built by 
> the remote daemon, plus fixed-output derivations.

One thing that does not require mutual trust, roughly like you describe
is:

  GUIX_DAEMON_SOCKET=ssh://guix.example.org guix build …

We could have an HTTP bridge and that’d be workable.  It could be just
streaming the daemon RPCs as-is on websockets, or defining an HTTP API
for each useful RPC.

Perhaps some of this can be also addressed with the Guix Build
Coordinator, which already provides an HTTP API, although a higher-level
one.  Chris?

>> WDYT? How does everyone else handle big builds? Do you have access
>> to
>> powerful workstations?

I have a 4-core Intel i7 laptop, which is okay for many things, and I
also have access to a couple of 32-core machines when I need to test
bigger builds like GCC.  And then there’s waiting for ci.guix feedback.

Ludo’.

Re: Public guix offload server

[Ludovic Courtès]

Arun Isaac <arunisaac@systemreboot.net> skribis:

> I just realized we might already have something close to this second,
> less powerful offload protocol that needs only one-way trust. According
> to the NEWS file, since Guix 0.13.0, the GUIX_DAEMON_SOCKET environment
> variable lets us specify remote daemons. See "(guix) The Store" in the
> manual for detailed documentation. The only thing missing is some way to
> retrieve the built output from the remote store.

We could simply run ‘guix publish’ on those servers.

Ludo’.

Re: Public guix offload server

[Ludovic Courtès]

Hi!

Vagrant Cascadian <vagrant@debian.org> skribis:

> I wonder if OSUOSL (or maybe other organizations) would be willing to
> provide a nice big server with access restricted to guix committers or
> something?
>
>   https://osuosl.org/services/hosting/
>
> I know they provide some very capable machines for reproducible builds
> and numerous other projects... (machines capable of running many virtual
> machines, FWIW)
>
> (And some POWER9 machines for guix, if I recall...)

Yes, that sounds like a good option.

Would someone like to contact them on behalf of the project, Cc:
guix-sysadmin?  :-)

Thanks,
Ludo’.

Re: Public guix offload server

[Tobias Geerinckx-Rice]

[-- Attachment #1: Type: text/plain, Size: 210 bytes --]

Ludovic Courtès 写道：
> Would someone like to contact them on behalf of the project, Cc:
> guix-sysadmin?  :-)

I'll do it.  They know^Wvaguely remember me from our guix-p9.

Kind regards,

T G-R

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 247 bytes --]