From mboxrd@z Thu Jan 1 00:00:00 1970 From: Julien Lepiller Subject: Re: Feature requests Date: Sat, 23 Mar 2019 15:36:05 +0100 Message-ID: <43538A49-AEB8-418B-8467-4CB45B25A3D2@lepiller.eu> References: <6BE771AB-3457-4FE6-87C3-98CCC4166083@lepiller.eu> Mime-Version: 1.0 Content-Type: multipart/alternative; boundary="----8UPE7MURSG7U42C4V9GGZBCMC6NALU" Content-Transfer-Encoding: 7bit Return-path: Received: from eggs.gnu.org ([209.51.188.92]:36449) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1h7hkz-0006UL-Aa for guix-devel@gnu.org; Sat, 23 Mar 2019 10:36:23 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1h7hkx-0003oE-3V for guix-devel@gnu.org; Sat, 23 Mar 2019 10:36:21 -0400 Received: from lepiller.eu ([89.234.186.109]:47886) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1h7hkv-0003iB-Hn for guix-devel@gnu.org; Sat, 23 Mar 2019 10:36:19 -0400 In-Reply-To: List-Id: "Development of GNU Guix and the GNU System distribution." List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: guix-devel-bounces+gcggd-guix-devel=m.gmane.org@gnu.org Sender: "Guix-devel" To: Joshua Marshall Cc: guix-devel@gnu.org ------8UPE7MURSG7U42C4V9GGZBCMC6NALU Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mh=E2=80=A6 so I'm still not sure if that's an actual issue=2E You can alwa= ys install two software that rely on different versions of a library=2E I g= uess in your example it only means that we need two libc packages, right? A= s long as you're able to define a package recipe for everything you want, u= sing different versions of the same package is fine, because they are diffe= rent inputs, so they have a different store path=2E A package built with glibc 2=2E10 well reference that old libc and never k= now about the newer ones even if they're installed in the same profile, as = long as the libc is not a propagated-input=2E I still don't see an incompatibility that's not solved by guix=2E With pyt= hon packages however that's very different because inputs are propagated=2E= If python could reference its dependencies directly, it would have been ju= st fine, but indeed two incompatible dependencies will end up in your profi= le=2E We can wrap python software already though, so it ignores the environ= ment and picks only the dependencies it was built with=2E That's not always= the best solution though=2E Maybe you can give as a concrete example for t= he python case? Then we will be able to help you better and come up with th= e right solution for your problems :) From=20what you describe I still think that guix environments are what you w= ant: if you have python-foo and python-bar that both depend on incompatible= versions of python-baz, you can create an environment for python-foo: guix environment --ad-hoc python python-foo --pure In which you can use python-foo but not python-bar, and a similar command = will give you a similar environment where you can run python-bar but not py= thon-foo=2E If you suppose these python packages provide a foo and a bar command, you = could also do: alias bar=3Dguix environment --ad-hoc python python-bar --pure -- bar And the same for foo, and you would have isolated environments=2E You can = also use that to control what python-foo has access to at runtime=2E For in= stance if it can use python-foobar at runtime, you would do: alias foo=3Dguix environment --ad-hoc python python-foo python-foobar --pu= re -- foo And the foo command now has access to its runtime dependency (if it wasn't= already declared at build time)=2E HTH Le 23 mars 2019 15:01:24 GMT+01:00, Joshua Marshall a =C3=A9crit : >I'm still getting my feet wet as it were=2E As a more relatable example, >lets use glibc=2E Say an application developed in 2000 had bug >compatibility >with glibc at that time=2E Then say that bug is fixed circa 2005 and >another >introduced=2E Then have in 2010 an application made with bug >compatibility >for the 2005 version=2E Now, if both of these applications were >co-installed >at most one could work due to bug compatibility=2E Now I'm not saying >this >is good practice or even common, but this type of thing happens at my >work >frequently enough that we have a whole toolset dedicated to managing >just >this problem=2E It gets far more severe in the python world=2E To cope >with >such poorly behaved programs, greater levels of isolation are needed, >like >each individual thing running in its own Docker container=2E But >sometimes >even this isn't enough as the dependencies will have a similar problem >and >so such isolation measures need to be recursively applied=2E It gets >messy=2E > >On Fri, Mar 22, 2019 at 6:30 PM Julien Lepiller >wrote: > >> Hi! >> >> I'm not sure what you mean when you talk about incompatible packages, >> maybe you could give a concrete example? I don't think there's >anyching >> that couldn't go to the store at least=E2=80=A6 So you can always creat= e >separate >> profiles=2E >> >> That said, I think people are working on improving the network >support in >> guix environment containers, and I think it's a good thing :)=2E I'm >not sure >> about cgroups, but our environments already provide quite a bit of >> isolation=2E Have you tried "guix environment --ad-hoc python >--container" >> for instance? There are more options to control what parts of the >file >> system is available inside the container=2E >> >> Le 22 mars 2019 18:47:19 GMT+01:00, Joshua Marshall > >> a =C3=A9crit : >>> >>> Hello all, >>> >>> I was told in IRC to post my possibly unreasonable feature requests >here=2E >>> >>> I've been thinking more on what Guix might be able to do, and what >would >>> make it more useful for a few of my past jobs=2E I'd like to see it >take on >>> the ability to have a per-installation target cgroup, network >namespace, >>> and filesystem chroot settings set with defaults which are >overridable at >>> invocation=2E In this way, a user could install and use packages with >>> mutually incompatible dependencies (I talked about this with a few >people >>> on IRC) like what happens with python=2E If this kind of >functionality were >>> added, it would largely supplant Docker, virtualenv, pip, poetry, >apk, >>> pacman, and probably a few other tools at my company which are there >just >>> to handle this kind of frailness=2E From this, I could also see an >entry >>> point to adding build module support to start to replace tools like >Make, >>> CMake, Meson, Bazel, and so on=2E >>> >>> These expand the scope of Guix quite a bit, but I think these are >needed >>> for it to really feel logically complete=2E Does all this make sense? >>> >>> >>> Please be advised that this email may contain confidential >information=2E >>> If you are not the intended recipient, please notify us by email by >>> replying to the sender and delete this message=2E The sender disclaims >that >>> the content of this email constitutes an offer to enter into, or the >>> acceptance of, any agreement; provided that the foregoing does not >>> invalidate the binding effect of any digital or other electronic >>> reproduction of a manual signature that is included in any >attachment=2E >>> >>> >>> >>> >>> >> >> -- >> Envoy=C3=A9 de mon appareil Android avec Courriel K-9 Mail=2E Veuillez >excuser ma >> bri=C3=A8vet=C3=A9=2E >> > >--=20 > > > >Please be advised that this email may contain confidential information=2E > >If you are not the intended recipient, please notify us by email by=20 >replying to the sender and delete this message=2E The sender disclaims >that=20 >the content of this email constitutes an offer to enter into, or the=20 >acceptance of, any agreement; provided that the foregoing does not=20 >invalidate the binding effect of any digital or other electronic=20 >reproduction of a manual signature that is included in any attachment=2E > > >=20 >=C2=A0=C2=A0=20 >=C2=A0=C2=A0=20 > --=20 Envoy=C3=A9 de mon appareil Android avec Courriel K-9 Mail=2E Veuillez exc= user ma bri=C3=A8vet=C3=A9=2E ------8UPE7MURSG7U42C4V9GGZBCMC6NALU Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mh=E2=80=A6 so I'm still not sure if that's an act= ual issue=2E You can always install two software that rely on different ver= sions of a library=2E I guess in your example it only means that we need tw= o libc packages, right? As long as you're able to define a package recipe f= or everything you want, using different versions of the same package is fin= e, because they are different inputs, so they have a different store path= =2E

A package built with glibc 2=2E10 well reference that old libc a= nd never know about the newer ones even if they're installed in the same pr= ofile, as long as the libc is not a propagated-input=2E

I still don'= t see an incompatibility that's not solved by guix=2E With python packages = however that's very different because inputs are propagated=2E If python co= uld reference its dependencies directly, it would have been just fine, but = indeed two incompatible dependencies will end up in your profile=2E We can = wrap python software already though, so it ignores the environment and pick= s only the dependencies it was built with=2E That's not always the best sol= ution though=2E Maybe you can give as a concrete example for the python cas= e? Then we will be able to help you better and come up with the right solut= ion for your problems :)

From what you describe I still think that g= uix environments are what you want: if you have python-foo and python-bar t= hat both depend on incompatible versions of python-baz, you can create an e= nvironment for python-foo:

guix environment --ad-hoc python python-f= oo --pure

In which you can use python-foo but not python-bar, and a = similar command will give you a similar environment where you can run pytho= n-bar but not python-foo=2E

If you suppose these python packages pro= vide a foo and a bar command, you could also do:

alias bar=3Dguix en= vironment --ad-hoc python python-bar --pure -- bar

And the same for = foo, and you would have isolated environments=2E You can also use that to c= ontrol what python-foo has access to at runtime=2E For instance if it can u= se python-foobar at runtime, you would do:

alias foo=3Dguix environm= ent --ad-hoc python python-foo python-foobar --pure -- foo

And the f= oo command now has access to its runtime dependency (if it wasn't already d= eclared at build time)=2E

HTH

Le 2= 3 mars 2019 15:01:24 GMT+01:00, Joshua Marshall <j=2Emarshall@arroyo=2Ei= o> a =C3=A9crit :
I'm still getting my feet wet as it were=2E  As a mo= re relatable example, lets use glibc=2E  Say an application developed = in 2000 had bug compatibility with glibc at that time=2E  Then say tha= t bug is fixed circa 2005 and another introduced=2E  Then have in 2010= an application made with bug compatibility for the 2005 version=2E  N= ow, if both of these applications were co-installed at most one could work = due to bug compatibility=2E  Now I'm not saying this is good practice = or even common, but this type of thing happens at my work frequently enough= that we have a whole toolset dedicated to managing just this problem=2E&nb= sp; It gets far more severe in the python world=2E  To cope with such = poorly behaved programs, greater levels of isolation are needed, like each = individual thing running in its own Docker container=2E  But sometimes= even this isn't enough as the dependencies will have a similar problem and= so such isolation measures need to be recursively applied=2E  It gets= messy=2E

On Fri, Mar 22, 2019 at 6:30 PM Julien Lepiller <julien@lepiller=2Eeu> wrote:
=
Hi!

I'm not= sure what you mean when you talk about incompatible packages, maybe you co= uld give a concrete example? I don't think there's anyching that couldn't g= o to the store at least=E2=80=A6 So you can always create separate profiles= =2E

That said, I think people are working on improving the network s= upport in guix environment containers, and I think it's a good thing :)=2E = I'm not sure about cgroups, but our environments already provide quite a bi= t of isolation=2E Have you tried "guix environment --ad-hoc python --contai= ner" for instance? There are more options to control what parts of the file= system is available inside the container=2E

Le 22 mars 2019 18:47:19 GMT+01:00, Joshua Marshall <j=2Emarshall@arroyo=2Eio= > a =C3=A9crit :
Hello all,

I was told in IR= C to post my possibly unreasonable feature requests here=2E

<= /div>
I've been thinking more on what Guix might be able to do, and wha= t would make it more useful for a few of my past jobs=2E  I'd like to = see it take on the ability to have a per-installation target cgroup, networ= k namespace, and filesystem chroot settings set with defaults which are ove= rridable at invocation=2E  In this way, a user could install and use p= ackages with mutually incompatible dependencies (I talked about this with a= few people on IRC) like what happens with python=2E  If this kind of = functionality were added, it would largely supplant Docker, virtualenv, pip= , poetry, apk, pacman, and probably a few other tools at my company which a= re there just to handle this kind of frailness=2E  From this, I could = also see an entry point to adding build module support to start to replace = tools like Make, CMake, Meson, Bazel, and so on=2E

These expand the scope of Guix quite a bit, but I think these are needed f= or it to really feel logically complete=2E  Does all this make sense?<= br>


Please be advised that this em= ail may conta= in confidential information=2E If you are not the intended recipient, pleas= e notify us by email by replying to the sender and delete this message=2E T= he sender disclaims that the content of this email constitutes an offer to = enter into, or the acceptance of, any agreement; provided that the foregoin= g does not invalidate the binding effect of any digital or other electronic= reproduction of a manual signature that is included in any attachment=2E

    
=
--
Envoy=C3=A9 de mon appareil Android avec Courriel K-9 Mail=2E Ve= uillez excuser ma bri=C3=A8vet=C3=A9=2E


Please be advised that this em= ail may conta= in confidential information=2E If you are not the intended recipient, pleas= e notify us by email by replying to the sender and delete this message=2E T= he sender disclaims that the content of this email constitutes an offer to = enter into, or the acceptance of, any agreement; provided that the foregoin= g does not invalidate the binding effect of any digital or other electronic= reproduction of a manual signature that is included in any attachment=2E

    
<= br>--
Envoy=C3=A9 de mon appareil Android avec Courriel K-9 Mail=2E Veu= illez excuser ma bri=C3=A8vet=C3=A9=2E ------8UPE7MURSG7U42C4V9GGZBCMC6NALU--